A HMM%2DBased Approach to Modeling Ant Behavior.

Generative Model versus Discriminative Model approach for Symbolic Sequence Classification.

Modeling Ant Activity by Means of Structured HMMs.

Structured Hidden Markov Models A General Tool for Modeling Agent Behaviors.

Structured Hidden Markov Model A General Framework for Modeling Complex Sequences.

Modeling Temporal Behavior via Structured Hidden Markov Models an Application to Keystroking Dynamics.

Incremental Construction of Structured Hidden Markov Models.

Hierarchical Hidden Markov Models for User%2FProcess Profile Learning.

Machine Learning and Data Mining.
this article, we focus primarily on the role of machine learning algorithms in the data mining process. The above medical example thus represents a prototypical data mining problem in which the data consists of a collection of time series descriptions, and we use the data to learn to predict later events in the series (emergency C-section) based on earlier events (symptoms before delivery). Although we used a medical example for illustration, we could have given an analogous example of learning to predict which bank loan applicants are at high risk of failing to repay their loan (see Figure 2). As shown in the figure, in this application data typically consists of time series descriptions of customers&#039; bank balances and other demographic information, rather than medical symptoms. Yet other applications, shown in Figure 3, include predicting customer purchase behavior, customer retention, and the quality of goods produced on a particular manufacturing line. All of these are applications for which data mining has been successfully applied in practice, and where further research promises yet more effective techniques. 3 The State of the Art, and What Next?
Experimental Evaluation of Hierarchical Hidden Markov Models.

Learning Complex Event Descriptions by Abstraction.

Learning Profiles Based on Hierarchical Hidden Markov Model.

Learning User Profile from Traces.

Learning Regular Expressions from Noisy Sequences.

Learning Complex and Sparse Events in Long Sequences.

Transition de phase et apprentissage relationnel  une analyse critique.

Relational Learning as Search in a Critical Region.
Machine learning strongly relies on the covering test to assess whether a candidate hypothesis covers  training examples. The present paper investigates learning relational concepts from examples,  termed relational learning or inductive logic programming. In particular, it investigates the chances  of success and the computational cost of relational learning, which appears to be severely affected  by the presence of a phase transition in the covering test. To this aim, three up-to-date relational  learners have been applied to a wide range of artificial, fully relational learning problems. A first  experimental observation is that the phase transition behaves as an attractor for relational learning;  no matter which region the learning problem belongs to, all three learners produce hypotheses  lying within or close to the phase transition region. Second, a failure region appears. All three  learners fail to learn any accurate hypothesis in this region. Quite surprisingly, the probability of  failure does not systematically increase with the size of the underlying target concept: under some  circumstances, longer concepts may be easier to accurately approximate than shorter ones. Some  interpretations for these findings are proposed and discussed.
Recognizing and Discovering Complex Events in Sequences.

A Monte Carlo Approach to Hard Relational Learning Problems.

Learning on the Phase Transition Edge.

Analyzing Relational Learning in the Phase Transition Framework.

Can Relational Learning Scale Up%3F

Phase Transitions in Relational Learning.
Machine learning strongly relies on the covering test to assess whether a candidate hypothesis covers  training examples. The present paper investigates learning relational concepts from examples,  termed relational learning or inductive logic programming. In particular, it investigates the chances  of success and the computational cost of relational learning, which appears to be severely affected  by the presence of a phase transition in the covering test. To this aim, three up-to-date relational  learners have been applied to a wide range of artificial, fully relational learning problems. A first  experimental observation is that the phase transition behaves as an attractor for relational learning;  no matter which region the learning problem belongs to, all three learners produce hypotheses  lying within or close to the phase transition region. Second, a failure region appears. All three  learners fail to learn any accurate hypothesis in this region. Quite surprisingly, the probability of  failure does not systematically increase with the size of the underlying target concept: under some  circumstances, longer concepts may be easier to accurately approximate than shorter ones. Some  interpretations for these findings are proposed and discussed.
Relational Learning Hard Problems and Phase Transitions.

On Some Misbehaviour of Back%2DPropagation with Non%2DNormalized RBFNs and a Solution.

An Experimental Study of Phase Transitions in Matching.

High%2DPerformance Knowledge Extraction from Data on PC%2DBased Networks of Workstations.
. The automatic construction of classifiers (programs able to  correctly classify data collected from the real world) is one of the major  problems in pattern recognition and in a wide area related to Artificial  Intelligence, including Data Mining. In this paper we present G-Net, a  distributed algorithm able to infer classifiers from pre-collected data, and  its implementation on PC-based Networks of Workstations (PC-NOWs).  In order to effectively exploit the computing power provided by PCNOWs,  G-Net incorporates a set of dynamic load distribution techniques  that allow it to adapt its behavior to variations in the computing power  due to resource contention. Moreover, it is provided with a fault tolerance  scheme that enables it to continue its computation even if the majority  of the machines become unavailable during its execution.  1 Introduction  The induction of classification rules from a database of instances pre-classified by a teacher is a task widely investigated both in...
High%2DPerformance Data Mining on Networks of Workstations.

On%2DLine Estimation of Matching Complexity in First Order Logic.

An Integrated Framework of Learning Numerical Terms in FOL.

Coevolutionary%2C Distributed Search for Inducing Concept Description.

An Experimental Evaluation of Coevolutive Concept Learning.
In this paper an extensive experimental evaluation of an evolutionary approach to concept learning is presented. The experimentation, performed with the system G-NET, investigates the effectiveness of the approach along the following dimensions: Robustness with respect to parameter setting, effectiveness of the MDL criterion coupled with a stochastic search bias, impact of coevolution on the quality of the solution and on the computational effort required, and ability to face problems requiring structured representation languages. A discussion of the obtained results and a suggestion on when this type of approach might be useful is also provided. 1 INTRODUCTION  Supervised concept learning has been tackled, so far, with several approaches, including symbolic, connectionist and evolutive ones. Different approaches are better suited to different classes of problems, depending, for instance, on the nature of data or the availability of domain-specific knowledge. In the hope of making a li...
Learning Classification Programs The Genetic Algorithm Approach.

Refining Numerical Terms in Horn Clauses.
In this paper an extensive experimental evaluation of an evolutionary approach to concept learning is presented. The experimentation, performed with the system G-NET, investigates the effectiveness of the approach along the following dimensions: Robustness with respect to parameter setting, effectiveness of the MDL criterion coupled with a stochastic search bias, impact of coevolution on the quality of the solution and on the computational effort required, and ability to face problems requiring structured representation languages. A discussion of the obtained results and a suggestion on when this type of approach might be useful is also provided. 1 INTRODUCTION  Supervised concept learning has been tackled, so far, with several approaches, including symbolic, connectionist and evolutive ones. Different approaches are better suited to different classes of problems, depending, for instance, on the nature of data or the availability of domain-specific knowledge. In the hope of making a li...
A Network Genetic Algorithm for Concept Learning.
This paper presents a highly parallel genetic algorithm, designed for concept induction in propositional and first order logics. The system exploits niches and species for learning multimodal concepts; it deeply differs from other systems because of the distributed architecture, which totally eliminates the concept of common memory. A first implementation of the system, designed for checking the possibility of exploiting parallel processing in network computer, is evaluated on standard benchmarks. The experimental results show that the system reaches good performances both with respect to the quality of the learned knowledge and with respect to the speed up on a workstation cluster. 1 INTRODUCTION  In the recent literature, Genetic Algorithms (GAs) emerged as valuable search tools in the field of concept induction (De Jong et al., 1993; Janikow, 1993; Greene &amp; Smith, 1993; Giordana &amp; Neri, 1996). Specifically, two of their features look particularly attractive, namely the exploration p...
A Coevolutionary Approach to Concept Learning.

FONN Combining First Order Logic with Connectionist Learning.
This paper presents a neural network architecture that can manage structured data and refine knowledge bases expressed in a first order logic language. The presented framework is well suited to classification problems in which concept descriptions depend upon numerical features of the data. In fact, the main goal of the neural architecture is that of refining the numerical part of the knowledge base, without changing its structure. In particular, we discuss a method to translate a set of classification rules into neural computation units. Here, we focus our attention on the translation method and on algorithms to refine network weights on structured data. The classification theory to be refined can be manually handcrafted or automatically acquired by a symbolic relational learning system able to deal with numerical features. As a matter of fact, the primary goal is to bring into a neural network architecture the capability of dealing with structured data of unrestricted size, by allowi...
A Coevolutionary Approach to Concept Learning.

Refining First Order Theories with Neural Networks.

Integrating Multiple Learning Strategies in First Order Logics.
. This paper describes a representation framework, that offers a unifying platform for alternative systems, which learn concepts in First Order Logic. The main aspects of this framework are discussed. First of all, the separation between the hypothesis logical language (a version of the  V L21 language) and the representation of data by means of a relational database is motivated. Then, the functional layer between data and hypotheses, which makes the data accessible by the logical level through a set of abstract properties. A novelty, in the hypothesis representation language, is the introduction of the construct of internal disjunction; such a construct, first used by the AQ and Induce systems, is here made operational via a set of algorithms, capable to learn it, for both the discrete and the continuous-valued attributes case. These algorithms are embedded in learning systems (SMART+, REGAL, SNAP, WHY, RTL) using different paradigms (symbolic, genetic or connectionist), thus realizi...
A Fine%2DGrained Parallel Evolutionary Program for Concept Induction.

Genetic Algorithms in Machine Learning.
One approach to the design of learning systems is to extract heuristics from existing adaptive systems. Genetic algorithms are heuristic learning models based on principles drawn from natural evolution and selective breeding. Some features that distinguish genetic algo- rithms from other search methods are:   A population of structures that can be interpreted as candidate solutions to the given problem;   The competitive selection of structures for reproduction, based on each structure&#039;s fitness as a solution to the given problem;   Idealized genetic operators that alter the selected structures in order to create new structures for fur- ther testing.
Learning Controllers for Industrial Robots.
. One of the most significant cost factors in robotics applications is the design and development of real-time robot control software. Control theory helps when linear controllers have to be developed, but it doesn&#039;t sufficiently support the generation of non-linear controllers, although in many cases (such as in compliance control), nonlinear control is essential for achieving high performance. This paper discusses how Machine Learning has been applied to the design of (non-)linear controllers. Several alternative function approximators, including Multilayer Perceptrons (MLP), Radial Basis Function Networks (RBFNs), and Fuzzy Controllers are analyzed and compared, leading to the definition of two major families: Open Field Function Function Approximators and Locally Receptive Field Function Approximators. It is shown that RBFNs and Fuzzy Controllers bear strong similarities, and that both have a symbolic interpretation. This characteristics allows for applying both symbolic and statis...
A Parallel Genetic Algorithm for Concept Learning.

Mapping Symbolic Knowledge into Locally Receptive Field Networks.
. This paper investigates Locally Receptive Field Networks, a broad class of neural networks including Probabilistic Neural Networks and Radial Basis Function Networks, which naturally exhibit symbolic properties. Moreover, specific attention is given to the sub-class of Factorizable Radial Basis Function Networks whose architecture can be directly translated into a propositional theory and viceversa. Exploiting this characteristics, symbolic and numeric algorithms can be easely integrated for automating network synthesis. Several methods including classification and regression trees, and statistical clustering are evaluated on a classification task in a difficult medical domain. The obtained results show that the considered network class is able to achieve a high accuracy, while conserving a symbolic readability. 1 Introduction  The problem of converting a body of symbolic knowledge into a neural network and viceversa has been already investigated by several authors. A major contribut...
A Parallel Genetic Algorithm for Concept Learning.

Search%2DIntensive Concept Induction.
This paper describes REGAL, a distributed genetic algorithm-based system, designed for learning First Order Logic concept descriptions from examples. The system is a hybrid between the Pittsburgh and the Michigan approaches, as the population constitutes a redundant set of partial concept descriptions, each evolved separately. In order to increase effectiveness, REGAL is specifically tailored to the concept learning task; hence, REGAL is task-dependent, but, on the other hand, domain-independent. The system proved to be particularly robust with respect to parameter setting across a variety of different application domains. REGAL is based on a selection operator, called Universal Suffrage operator, provably allowing the population to asymptotically converge, in average, to an equilibrium state, in which several species coexist. The system is presented both in a serial and in a parallel version, and a new distributed computational model is proposed and discussed. The system has been test...
Learning Disjunctive Concept Definitions Using a Genetic Algorithm.

Learning Disjunctive Concepts by Means of Genetic Algorithms.
be, 1969] Watanabe, S. (1969). Knowing and Guessing - A Formal and Qualitative  Study. Wiley, New York, NY.  [Weiss and Indurkhya, 1994] Weiss, S. M. and Indurkhya, N. (1994). Small sample decision tree pruning. In 11th International Machine Learning Conference, New Brunswick, NJ.  [Wilson, 1987] Wilson, S. (1987). Classifier systems and the animat problem. Machine Learning, 2:199--228.  [Wittgenstein, 1953] Wittgenstein, L. (1953). In Ascombe, G. E. M., editor, Philosophical  Investigations. Oxford: Blackwell, London, UK.  [Wnek and Michalski, 1994] Wnek, J. and Michalski, R. (1994). Hypothesis-driven constructive induction in aq17-hci: A method and experiments. Machine Learning, 14:136--247.  BIBLIOGRAPHY 159 of the 8th International Workshop on Machine Learning, pages 203--207, Evanston, IL. Morgan Kaufmann.  [Slotta and Chi, 1996] Slotta, J. and Chi, M. T. H. (1996). Understandin
Formal Models of Selection in Genetic Algorithms.
be, 1969] Watanabe, S. (1969). Knowing and Guessing - A Formal and Qualitative  Study. Wiley, New York, NY.  [Weiss and Indurkhya, 1994] Weiss, S. M. and Indurkhya, N. (1994). Small sample decision tree pruning. In 11th International Machine Learning Conference, New Brunswick, NJ.  [Wilson, 1987] Wilson, S. (1987). Classifier systems and the animat problem. Machine Learning, 2:199--228.  [Wittgenstein, 1953] Wittgenstein, L. (1953). In Ascombe, G. E. M., editor, Philosophical  Investigations. Oxford: Blackwell, London, UK.  [Wnek and Michalski, 1994] Wnek, J. and Michalski, R. (1994). Hypothesis-driven constructive induction in aq17-hci: A method and experiments. Machine Learning, 14:136--247.  BIBLIOGRAPHY 159 of the 8th International Workshop on Machine Learning, pages 203--207, Evanston, IL. Morgan Kaufmann.  [Slotta and Chi, 1996] Slotta, J. and Chi, M. T. H. (1996). Understandin
Learning Disjunctive Concepts with Distributed Genetic Algorithms.
this paper, REGAL [Giordana &amp; Saitta, 1993], encodes first order logic (FOL) formulas j into bit strings of fixed length. In order to make this possible, REGAL&#039;s representation language, L, has been constrained to be finite, by defining a language template, L, which represents the maximally complex formula in L. The resulting language contains FOL formulas built up with conjunction, disjunction, negation, internal disjunction and existential and universal quantification. In REGAL five genetic operators have been defined: the two-point and the uniform crossovers [De Jong, 1975; Goldberg, 1989], the generalizing and specializing crossovers, specifically designed for the task at hand [Giordana &amp; Sale, 1992], and the seeding operator, primarily used to initialize the population, in order to start with a set of formulas, each covering at least one example of the concept h [Giordana &amp; Saitta, 1993]. In the basic genetic algorithm an improved version of the sharing functions mechanism [Goldberg &amp; Richardson, 1987] has been introduced, in order to allow the formation of subpopulations. This is useful when concepts have a multimodal structure, as will be discussed in the following sections. 2. Learning Disjunctive Concepts The fundamental novelty of REGAL is its ability to learn disjunctive concepts. A disjunctive concept requires a description of the form: j 1 Ú j 2 Ú .... Ú j m ® h, (2) where each formula j j (1  j  m) covers a region of the instance space and corresponds to a different mode of the concept. If the fitness function f(j), associated to a formula j, takes into account, as usual, completeness and consistency [Michalski, 1983], then f(j) is likely to exhibit several maxima, in correspondance to the modes of the concept. + In presence of noise these strict condition...
Learning Relations Using Genetic Algorithms.
be, 1969] Watanabe, S. (1969). Knowing and Guessing - A Formal and Qualitative  Study. Wiley, New York, NY.  [Weiss and Indurkhya, 1994] Weiss, S. M. and Indurkhya, N. (1994). Small sample decision tree pruning. In 11th International Machine Learning Conference, New Brunswick, NJ.  [Wilson, 1987] Wilson, S. (1987). Classifier systems and the animat problem. Machine Learning, 2:199--228.  [Wittgenstein, 1953] Wittgenstein, L. (1953). In Ascombe, G. E. M., editor, Philosophical  Investigations. Oxford: Blackwell, London, UK.  [Wnek and Michalski, 1994] Wnek, J. and Michalski, R. (1994). Hypothesis-driven constructive induction in aq17-hci: A method and experiments. Machine Learning, 14:136--247.  BIBLIOGRAPHY 159 of the 8th International Workshop on Machine Learning, pages 203--207, Evanston, IL. Morgan Kaufmann.  [Slotta and Chi, 1996] Slotta, J. and Chi, M. T. H. (1996). Understandin
SMART  A Multi%2DStrategy Learning Tool.
Inducing concept descriptions in First Order Logic is inherently a complex task. There are two main reasons: on one hand, the task is usually formulated as a search problem inside a very large space of logical descriptions which needs strong heuristics to be kept to manageable size. On the other hand, most developed algorithms are unable to handle numerical features, typically occurring in realworld data. In this paper, we describe the learning system SMART+, that embeds sophisticated knowledge-based heuristics to control the search process and is able to deal with numerical features. SMART+ can use different learning strategies, such as inductive, deductive and abductive ones, and exploits both backgruond knowledge and statistical evaluation criteria. Furthermore, it can use simple Genetic Algorithms to refine predicate semantics and this aspect will be described in detail. Finally, an evaluation of SMART+ performances is made on a complex task. 1 Introduction In the recent literature...

Learning Simple Recursive Theories.
Induction of recursive theories in the normal ILP setting is a difficult learning task whose  complexity is equivalent to multiple predicate learning. In this paper we propose computational  solutions to some relevant issues raised by the multiple predicate learning problem. A separate-andparallel  -conquer search strategy is adopted to interleave the learning of clauses supplying predicates  with mutually recursive definitions. A novel generality order to be imposed on the search space  of clauses is investigated, in order to cope with recursion in a more suitable way. The consistency  recovery is performed by reformulating the current theory and by applying a layering technique,  based on the collapsed dependency graph. The proposed approach has been implemented in the  ILP system ATRE and tested on some laboratory-sized and real-world data sets. Experimental results  demonstrate that ATRE is able to learn correct theories autonomously and to discover concept  dependencies. Finally, related works and their main differences with our approach are discussed.
Learning Relations An Evaluation of Search Strategies.
. Inducing concept descriptions in first order logic is inherently a complex task; then, heuristics are needed to keep the problem to manageable size. In this paper we explore the effect of alternative search strategies, including the use of information gain and of a-priori knowledge, on the quality of the acquired relations, intended as the ability to reconstruct the rule used to generate the examples. To this aim, an artificial domain has been created, in which the experimental conditions can be kept under control, the &#034;solution&#034; of the learning problem is known and a perfect theory is available. Another investigated aspect is the impact of more complex description languages, such as, for instance, including numerical quantifiers. The results show that the information gain criterion is too greedy to be useful when the concepts have a complex internal structure; however, this drawback is more or less shared with any purely statistical evaluation criterion. The addition of parts of the...

Comparison of Search Strategies in Learning Relations.

Learning Structured Concepts Using Genetic Algorithms.

Learning Mutually Dependent Relations.

Extending Inverse Resolution to Build Up Abstractions.

Abstracting Background Knowledge for Concept Learning.
ion in problem solving and learning. In Proceedings of the International Join Conference on Artifical Intelligence-89, pages 590--595, Detroit, MI, 1989. Morgan Kaufmann. [ Vorberg and Goebel, 1991 ] D. Vorberg and R. Goebel. Das losen rekursiver programmierprobleme: Rekursionsschemata. Kognitionswissenschaft, 1:83--95, 1991. [ Yoo and Fisher, 1991 ] J. Yoo and D. Fisher. Concept formation over explanations and problem-solving experience. In J. Mylopoulos and R. Reiter, editors, Proceedings of the Twelfth International Conference on Artificial Intelligence, volume 2, pages 630--637, San Mateo, CA, 1991. Morgan Kaufmann.  [ Michalski and Kodratoff, 1990 ] R. S. Michalski and Y. Kodratoff. Research in machine learning: Recent progress, classification of methods, and future directions. In Y. Kodratoff and R. S. Michalski, editors, Machine learning: An artificial intelligence approach,  volume 3, chapter 1, pages 3--30. Morgan Kaufmann, San Mateo, CA, 1990. [ Minton et al., 1989 ] S. Mint...
Learning Quantitative Features in a Symbolic Environment.

Investigating the Relationships between Abduction and Inverse Resolution in Proposition Calculus.

Abstracting Concepts with Inverse Resolution.

Integrated Learning in a Real Domain.

Biasing Induction by Using a Domain Theory An Experimental Evaluation.

Integrated Learning in a real Domain.


Deduction in Top%2DDown Inductive Learning.

Knowledge representation and use in pattern analysis.


A Knowledge Intensive Approach to Concept Induction.

Automated Concept Acquisition in Noisy Environments.

Integrating EBL and SBL Approaches to Knowledge Base Refinement.

Discrimination of Words in a Large Vocabulary Using Phonetic Descriptions.

Modelling Production Systems by means of Predicate%2DTransitions Networks.

An expert system oriented to complex pattern recognition problems.


An expert system for mapping acoustic cues into phonetic features.


An Expert System for Speech Decoding.

